Information processing device, information processing method, and program

ABSTRACT

The present technology relates to an information processing device, an information processing method, and a program capable of allowing a user viewing a video of a remote location to be aware of presence of a person who is outside of an angle of field of a video in more appropriate form.An information processing device according to the present technology includes a processing unit, wherein, when, between a first space where a first imaging device and a first display device are installed and a second space where a second imaging device and a second display device are installed, a captured image captured by the imaging device in one of the spaces is displayed by the display device in the other space in real time, the processing unit performs processing to notify first users in the first space that second users are outside of an angle of field of the second imaging device when the second users who are at positions outside of the angle of field of the second imaging device is detected. The present technology can be applied to, for example, a video communication system.

TECHNICAL FIELD

The present technology relates to an information processing device, aninformation processing method, and a program, and more particularly, toan information processing device, an information processing method, anda program capable of allowing a user viewing a video of a remotelocation to be aware of the presence of people who are outside of anangle of field of the video in more appropriate form.

BACKGROUND ART

In the related art, users at remote locations can talk while viewingeach other's faces with video communication systems, and thus morefamiliar communication can be achieved.

For example, PTL 1 discloses a video display system in whichcommunication can be implemented in a sense of presence as if users atremote locations were actually looking at each other.

CITATION LIST Patent Literature [PTL 1]

-   JP 2016-192686 A

SUMMARY Technical Problem

Incidentally, in the above-described video display system, users viewingscenes of remote locations through displays are sometimes unaware ofpeople who are outside of angles of fields of cameras used to capturevideos of the remote locations.

In particular, when there are a plurality of people viewing aspects ofremote locations, only some users paying attention to an angle of fieldoutside of the angle of field are preferably allowed to be aware ofpeople outside of the angle of field.

The present technology has been devised in view of such circumstancesand allows users viewing videos of remote locations to be aware of thepresence of a person who is outside of an angle of field of the videosin a more appropriate form.

Solution to Problem

An information processing device according to an aspect of the presenttechnology is an information processing device including a processingunit. Between a first space where a first imaging device and a firstdisplay device are installed and a second space where a second imagingdevice and a second display device are installed, when a captured imagecaptured by the imaging device in one of the spaces is displayed by thedisplay device in the other space in real time, the processing unitperforms processing to notify first users in the first space that secondusers are outside of an angle of field of the second imaging device whenthe second users who are at positions outside of the angle of field ofthe second imaging device are detected.

An information processing method according to an aspect of the presenttechnology is an information processing method of an informationprocessing device. Between a first space where a first imaging deviceand a first display device are installed and a second space where asecond imaging device and a second display device are installed, when acaptured image captured by the imaging device in one of the spaces isdisplayed by the display device in the other space in real time, theinformation processing device performs processing to notify first usersin the first space that second users are outside of an angle of field ofthe second imaging device when the second users who are at positionsoutside of the angle of field of the second imaging device are detected.

An information processing device according to still another aspect ofthe present technology is a program causing a computer to perform, when,between a first space where a first imaging device and a first displaydevice are installed and a second space where a second imaging deviceand a second display device are installed, a captured image captured bythe imaging device in one of the spaces is displayed by the displaydevice in the other space in real time, processing to notify first usersin the first space that second users are outside of an angle of field ofthe second imaging device when the second users who are at positionsoutside of the angle of field of the second imaging device are detected.

According to an aspect of the present technology, between a first spacewhere a first imaging device and a first display device are installedand a second space where a second imaging device and a second displaydevice are installed, when a captured image captured by the imagingdevice in one of the spaces is displayed by the display device in theother space in real time, processing is performed to notify first usersin the first space that second users are outside of an angle of field ofthe second imaging device when the second users who are at positionsoutside of the angle of field of the second imaging device are detected.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an exemplary configuration of a videocommunication system according to an embodiment of the presenttechnology.

FIG. 2 is a front view illustrating an exemplary configuration of theexterior of a telecommunication device.

FIG. 3 is a diagram illustrating an example of a situation in which thetelecommunication devices are used.

FIG. 4 is a diagram illustrating an overview of first output control ofthe telecommunication device.

FIG. 5 is a diagram illustrating an example of a combination of thedegree of interest of a user and output methods on a site A side.

FIG. 6 is a block diagram illustrating a first exemplary configurationof the telecommunication device.

FIG. 7 is a flowchart illustrating an operation of the telecommunicationdevice.

FIG. 8 is a diagram illustrating an overview of second output control ofthe telecommunication device.

FIG. 9 is a diagram illustrating an overview of second output control ofthe telecommunication device.

FIG. 10 is a diagram illustrating examples of combinations of situationsof a B site and output methods on the site A side.

FIG. 11 is a block diagram illustrating a second exemplary configurationof the telecommunication device.

FIG. 12 is a flowchart illustrating an operation of thetelecommunication device.

FIG. 13 is a diagram illustrating an overview of third output control ofthe telecommunication device.

FIG. 14 is a diagram illustrating an overview of third output control ofthe telecommunication device.

FIG. 15 is a diagram illustrating an overview of fourth output controlof the telecommunication device.

FIG. 16 is a block diagram illustrating a third exemplary configurationof the telecommunication device.

FIG. 17 is a flowchart illustrating an operation of thetelecommunication device.

FIG. 18 is a diagram illustrating an overview of other output controlperformed by the telecommunication device.

FIG. 19 is a front view illustrating an example of the telecommunicationdevice in which an illuminant is provided.

FIG. 20 is a diagram illustrating an example of a relation betweenintensity of light and the degree of interest.

FIG. 21 is a diagram schematically illustrating examples of forms of acamera and a sensor.

FIG. 22 is a diagram illustrating another exemplary configuration of thevideo communication system.

FIG. 23 is a diagram illustrating still another exemplary configurationof the video communication system.

FIG. 24 is a block diagram illustrating an exemplary hardwareconfiguration of a computer.

DESCRIPTION OF EMBODIMENTS

Hereinafter, modes for carrying out the present technology will bedescribed. The description will be made in the following order.

1. Video communication system2. First output control3. Second output control4. Third output control5. Fourth output control6. Modification examples

1. Video Communication System

FIG. 1 is a diagram illustrating an exemplary configuration of a videocommunication system according to an embodiment of the presenttechnology.

As illustrated in FIG. 1, a video communication system 1 is configuredsuch that telecommunication devices 11 a and 11 b which are twoinformation processing devices are connected via a network 12 such asthe Internet.

The telecommunication devices 11 a and 11 b are provided in differentspaces such as different buildings or different rooms. A user near thetelecommunication device 11 a and a user near the telecommunicationdevice 11 b, as illustrated in FIG. 1, are users who are at remotelocations from each other.

The telecommunication devices 11 a and 11 b have basically the sameconfiguration. As will be described below in detail, in thetelecommunication devices 11 a and 11 b, cameras that image surroundingaspects, microphones that collect surrounding sounds such asenvironmental sounds, speakers that output sounds, and the like areprovided in addition to displays that have large sizes.

Videos captured by the cameras, sounds collected by the microphones, andthe like are transmitted and received, for example, regularly in realtime between the telecommunication devices 11 a and 11 b whileconnection of both sides is established.

The telecommunication device 11 a displays videos captured by thetelecommunication device 11 b and outputs sounds collected by thetelecommunication device 11 b.

In the videos captured by the telecommunication device 11 b, an aspectof a space where the telecommunication device 11 b is installed isshown, including an aspect of the user of the telecommunication device11 b. Sounds collected by the telecommunication device 11 b includeenvironmental sounds of the space where the telecommunication device 11b is installed in addition to sounds of the user of thetelecommunication device 11 b.

Thus, for example, the user of the telecommunication device 11 a facesthe far side of the telecommunication device 11 a near the user andfeels as if the user of the telecommunication device 11 b is present.

Similarly, the telecommunication device 11 b displays videos captured bythe telecommunication device 11 a and outputs sounds collected by thetelecommunication device 11 a.

In the videos captured by the telecommunication device 11 a, an aspectof a space where the telecommunication device 11 a is installed isshown, including an aspect of the user of the telecommunication device11 a. Sounds collected by the telecommunication device 11 a includeenvironmental sounds of the space where the telecommunication device 11a is installed in addition to sounds of the user of thetelecommunication device 11 a.

Thus, for example, the user of the telecommunication device 11 b facesthe far side of the telecommunication device 11 b near the user andfeels as if the user of the telecommunication device 11 a is present.

The user of the telecommunication device 11 a can achieve naturalcommunication with the user of the telecommunication device 11 b as ifthe user of the telecommunication device 11 b is present in an adjacentspace.

Similarly, the user of the telecommunication device 11 b can achievenatural communication with the user of the telecommunication device 11 aas if the user of the telecommunication device 11 a is present in anadjacent space.

That is, the users of the telecommunication devices 11 a and 11 b canachieve smoother communication while feeling close to each other bycommunicating without active awareness.

Hereinafter, when it is not necessary to distinguish thetelecommunication devices 11 a and 11 b from each other, thetelecommunication devices 11 a and 11 b are collectively referred to asthe telecommunication devices 11. The same applies to other pairedconfigurations in this description.

FIG. 2 is a front view illustrating an exemplary configuration of theexterior of the telecommunication device 11 a.

As illustrated in FIG. 2, a narrow frame 21 remains on the front surfaceof the casing of the telecommunication device 11 a and a longitudinallylong rectangular display 22 configured as a liquid crystal display(LCD), an organic electro luminescence (EL) display, or the like isprovided.

A sensor unit 23 is provided via an instruction member (not illustrated)fixed to, for example, the frame 21 in the front of the display 22. Thesensor unit 23 is provided with a camera 24 and sensors 25-1 and 25-2which are two sensors.

Of the upper, lower, left, and right edges of the frame 21, a microphone26 is provided on the upper edge, and speakers 27-1 and 27-2 areprovided on the left and right edges.

The display 22 displays a video in accordance with captured imagescaptured by the telecommunication device 11 b based on video informationtransmitted from the telecommunication device 11 b.

The camera 24 images a space in front of the telecommunication device 11a. The video information indicating a video in accordance with capturedimages captured by the camera 24 is transmitted to the telecommunicationdevice 11 b.

The sensors 25-1 and 25-2 are configured by various types of sensorssuch as cameras or infrared sensors. The sensors 25-1 and 25-2 generateenvironmental information based on detection results of the sensors.

The environmental information is information including videos inaccordance with captured images captured by the cameras include in thesensors 25 and information acquired by infrared sensors. In the video inaccordance with the captured images captured by the camera included inthe sensor 25, an aspect outside of the angle of camera field, which isa range outside of an angle of imaging field of the camera 24, is alsoshown.

The environmental information including information outside of the angleof camera field is transmitted to the telecommunication device 11 b. Inthe telecommunication device 11 b, for example, a sound heard fromoutside of the angle of camera field (a sound for which a positionoutside of the angle of camera field is a sound source position) isanalyzed based on the environmental information.

The microphone 26 collects a sound of the space where thetelecommunication device 11 a is installed. Sound information indicatingthe sound collected by the microphone 26 is transmitted to thetelecommunication device 11 b.

The speakers 27-1 and 27-2 output the sound of the space where thetelecommunication device 11 b is installed based on the soundinformation transmitted from the telecommunication device 11 b.

The telecommunication device 11 b has the same configuration as thetelecommunication device 11 a illustrated in FIG. 2. Hereinafter, theconfiguration of the telecommunication device 11 a illustrated in FIG. 2is used as the configuration of the telecommunication device 11 b inthis description.

The positions at which the camera 24, the sensors 25-1 and 25-2, themicrophone 26, and the speakers 27-1 and 27-2 are installed in FIG. 2are exemplary, and may be other positions as long as the functions canbe implemented, and the numbers of installed sensors, microphones, andspeakers are arbitrary.

FIG. 3 is a diagram illustrating an example of a situation in which thetelecommunication devices 11 are used.

As illustrated in FIG. 3, for example, the telecommunication devices 11a and 11 b are installed at sites A and B which are mutually distantspaces. A wave line illustrated in the middle of FIG. 3 indicates thatthe space where the telecommunication device 11 a is installed is awayfrom the space where the telecommunication device 11 b is installed.

In the example of FIG. 3, at the site A, it is assumed that users A1 andA2 who are two users are in front of the telecommunication device 11 a.For example, the user A1 is a user who views a video displayed on thetelecommunication device 11 a and talks with a user B1 who is at thesite B. The user A2 is a user who views the telecommunication device 11a from the rear position of the user A1.

At the site B, on the other hand, it is assumed that users B1 and B2 whoare two users are in front of the telecommunication device 11 b. Forexample, the user B1 is a user who views a video displayed on thetelecommunication device 11 b and talks with the user A1 who is at thesite A. The user B2 is a user who is performing work outside of an angleof camera field.

Range #1 indicated by a dotted line as a range of the site B is a rangeof an angle of camera field of the telecommunication device 11 b (thecamera 24 installed in the telecommunication device 11 b). Ranges #2 and#3 indicated by dotted lines are ranges outside of the angle of camerafield of the telecommunication device 11 b (the camera 24 installed inthe telecommunication device 11 b).

In this way, the telecommunication devices 11 a and 11 b are used in asituation in which there are a plurality of users around in some cases.Outputs of the telecommunication devices 11 a and 11 b are controlled inaccordance with actions of the surrounding users.

2. First Output Control

Overview of Output Control

FIG. 4 is a diagram illustrating an overview of first output control ofthe telecommunication device 11.

In the example of FIG. 4, it is assumed that only the user A2 is infront of the telecommunication device 11 a at the site A. At the site B,on the other hand, it is assumed that there is only the user B2. Theuser B2 is a user who is within range #3 which is a range outside of theangle of camera field.

At this time, since there is nothing within range #1 which is a range ofthe angle of camera field of the telecommunication device 11 b, nothingis shown in a video displayed on the telecommunication device 11 a. Theuser A2 in front of the telecommunication device 11 a is assumed to beinterested in the person outside of the angle of camera field and toperform action of looking to see whether somebody is there, as indicatedby a balloon of FIG. 4.

In such a situation, when it is detected that the user A2 is performingan action of interest, the telecommunication device 11 a outputs anemphasized indication sound toward the user A2. The indication sound isa sound heard from outside of the angle of camera field so that the userA2 feels an indication of a person outside of the angle of camera fieldof the telecommunication device 11 b.

The action of interest is an action of the user who is paying attentionto the person outside of the angle of camera field. The action of a userwho is paying attention to the angle of field outside of the angle ofcamera field includes an action indicating interest in any of the leftand right ends of the display 22 of the telecommunication device 11 aand an action of paying attention to the person outside of the angle ofcamera field and looking into the telecommunication device 11 a.

The output of the indication sound is controlled in accordance withmagnitude of the degree of interest of the user A2 in the person outsideof the angle of camera field. The telecommunication device 11 a analyzesthe magnitude of the degree of interest based on the action of interestof the user A2 and controls an output method or the like for anindication sound in accordance with the magnitude of the degree ofinterest.

FIG. 5 is a diagram illustrating an example of a combination of thedegree of interest of the user A2 and output methods on a site A side.

In the example of FIG. 5, in descending order of the degree of interest,an action of gazing at an end of the display 22 on which a video of thesite B is displayed, an action of looking into an aspect of the site B,an action of checking whether there is a person outside of the angle ofcamera field at the site B by saying “Is anybody there?”, or an actionof turning a face toward the site B are shown as actions of interest ofthe user A2.

As shown in the second row of the lower column of the item in the firstrow of the table of FIG. 5, when the user A2 performs an action ofinterest of gazing at the end of the display 22 on which the video ofthe site B is displayed, the telecommunication device 11 a adjustsdirectivity of the speakers 27-1 and 27-2 and outputs an emphasizedindication sound toward the user A2. At this time, the telecommunicationdevice 11 a adjusts the directivity of the speakers 27-1 and 27-2 towardthe user A2 and performs processing for raising a volume of theindication sound.

As illustrated in the third row of the table of FIG. 5, when the user A2performs an action of interest of looking into the aspect of the site B,the telecommunication device 11 a adjusts the directivity of thespeakers 27-1 and 27-2 and outputs the indication sound toward the userA2. At this time, the telecommunication device 11 a performs processingfor adjusting the directivity of the speakers 27-1 and 27-2 toward theuser A2.

As illustrated in the fourth row of the table of FIG. 5, when the userA2 performs an action of interest of checking whether there is a personoutside of the angle of camera field in the B site by saying “Is anybodythere?”, the telecommunication device 11 a outputs the indication soundthrough an output method in response to presence or absence of aresponse to the speech of the user A2.

For example, when the user B2 responds, the telecommunication device 11a performs an output in a normal video communication system. As theoutput in the normal video communication system, for example, thetelecommunication device 11 a performs processing for outputting anenvironmental sound collected at the site B rather than the indicationsound heard from outside of the angle of camera field.

Conversely, when the user B2 does not respond, the telecommunicationdevice 11 a outputs the emphasized indication sound toward the wholesite A. At this time, the telecommunication device 11 a performsprocessing for raising a volume of the indication sound.

As illustrated in the fifth row of the table of FIG. 5, when the user A2views the display of the telecommunication device 11 a and performs theaction of interest of turning the face to the site B, thetelecommunication device 11 a performs an output in the normal videocommunication system.

In this way, as the degree of interest of the action of interest of theuser A2 is higher, the indication sound heard from outside of the angleof camera field is output toward the user A2 in a more emphasized form.

Thus, the user A2 can be aware of a person who is outside of the angleof camera field of the telecommunication device 11 b (the camera 24installed in the telecommunication device 11 b) of the connectiondestination by the indication sound output from the telecommunicationdevice 11 a.

Since the indication sound is output toward the user A2 performing theaction of interest, the telecommunication device 11 a can allow only theuser A2 paying attention to the angle of camera field outside of theangle of camera field of the connection destination to be aware ofpresence of the person who is outside of the angle of camera field.

For example, as in the user A1 of FIG. 3, the indication sound is notoutput to the user A1 talking with the user B1 who is not payingattention to the angle of field outside of the angle of camera field ofthe connection destination. The user A1 can continue to talk with theuser B1 without paying attention to the person outside of the angle ofcamera field.

In this way, since the indication sound is presented to only the userwho is paying attention to the person outside of the angle of camerafield, the user can be aware of the presence of the person outside ofthe angle of camera field in a more appropriate form.

Configuration of Telecommunication Device 11

FIG. 6 is a block diagram illustrating a first exemplary configurationof the telecommunication device 11.

As illustrated in FIG. 6, the telecommunication device 11 includes aninput unit 31, a processing unit 32, a transmission unit 33, a receptionunit 34, and an output unit 35.

The input unit 31 is configured by an imaging device such as the camera24, a sensing device such as the sensors 25-1 and 25-2, or a soundcollection device such as the microphone 26 in FIG. 2. For example, theinput unit 31 supplies the processing unit 32 with video information ofa space in accordance with a captured image captured by the camera 24,environmental information generated by the sensors 25-1 and 25-2, andsound information of an environmental sound collected by the microphone26.

A microphone collecting sounds outside of the angle of camera field anda depth sensor or the like measuring a distance to a subject imaged bythe camera 24 may be included as the sensors 25-1 and 25-2 in the inputunit 31. In this case, the environmental information can also include adetection result by the microphone or the depth sensor.

The processing unit 32 is configured by a central processing unit (CPU),a read-only memory (ROM), a random access memory (RAM), or the like. Theprocessing unit 32 controls an operation of the whole telecommunicationdevice 11 by executing a predetermined program.

As illustrated in FIG. 6, the processing unit 32 includes adegree-of-interest analysis unit 41, an environmental informationanalysis unit 42, an indication sound output method selection unit 43,and an indication sound control unit 44.

The degree-of-interest analysis unit 41 supplies the transmission unit33 with the video information, the environmental information, and thesound information of the environmental sound supplied from the inputunit 31.

The degree-of-interest analysis unit 41 analyzes the degree of interestof a user in front of the telecommunication device 11 in the personoutside of the angle of camera field of the connection destination basedon the video information, the environmental information, and the soundinformation of the environmental sound supplied from the input unit 31.

For example, the degree-of-interest analysis unit 41 analyzes magnitudeof the degree of interest of the user based on a position at which theuser in front of the telecommunication device 11 is gazing, a length ofa time in which a visual line is oriented to the display 22, a spokensound, and an orientation of the head of a user. As a method ofanalyzing the degree of interest in the person outside of the angle ofcamera field of the connection destination, any of various methods canbe used.

Degree-of-interest information which is information indicating thedegree of interest of the user analyzed by the degree-of-interestanalysis unit 41 is supplied to the indication sound output methodselection unit 43.

The environmental information analysis unit 42 supplies the output unit35 with the video information and the sound information of theenvironmental sound of the connection destination supplied from thereception unit 34.

The environmental information analysis unit 42 analyzes a sound heardfrom outside of the angle of camera field and a sound heard from withinthe angle of camera field which are included in the environmental soundof the connection destination based on the environmental information ofthe connection destination supplied from the reception unit 34. Thesound heard from outside of the angle of camera field is used as anindication sound which is appropriately presented to the user.

The environmental information analysis unit 42 that analyzes a soundheard from outside of the angle of camera field and a sound heard fromwithin the angle of camera field also detects a user who is outside ofthe angle of camera field of the connection destination.

Specifically, the environmental information analysis unit 42 detects auser who is outside of the angle of camera field by performing boneestimation using a video in which an aspect outside of the angle ofcamera field of the connection destination is shown. The bone estimationis a technology for estimating a skeleton of a person using a capturedimage. A user who is outside of the angel of camera field of theconnection destination may be detected by performing detection of abackground difference and face recognition using a video in which theaspect outside of the angle of camera field is shown.

The video in which the aspect outside of the angle of camera field ofthe connection destination is shown and which is used to detect the userwho is outside of the angle of camera field is information included inthe environmental information of the connection destination.

Information indicating an analysis result by the environmentalinformation analysis unit 42 is supplied to the indication sound outputmethod selection unit 43 along with the sound information of theenvironmental sound of the connection destination.

The indication sound output method selection unit 43 selects a method ofoutputting the indication sound based on the information supplied fromthe environmental information analysis unit 42 and thedegree-of-interest information supplied from the degree-of-interestanalysis unit 41. For example, the indication sound output methodselection unit 43 selects magnitude of a volume of the indication soundor directivity of the speakers 27-1 and 27-2 outputting the indicationsound.

The indication sound control unit 44 is supplied with the soundinformation of the environmental sound of the connection destination,the information indicating the result of the analysis by theenvironmental information analysis unit 42, and the informationindicating the output method for the indication sound.

The indication sound control unit 44 generates the sound information ofthe indication sound by processing the sound information of theenvironmental sound of the connection destination based on theinformation supplied from the indication sound output method selectionunit 43. For example, the indication sound control unit 44 generates thesound information of the indication sound by extracting the soundinformation of the indication sound heard from outside of the angle ofcamera field of the connection destination from the sound information ofthe environmental sound of the connection destination.

The indication sound control unit 44 supplies the sound information ofthe indication sound to the output unit 35 and causes the indicationsound to be output by the output method selected by the indication soundoutput method selection unit 43.

The transmission unit 33 transmits the video information, theenvironmental information, and the sound information of theenvironmental sound supplied from the processing unit 32 to thetelecommunication device 11 of the connection destination via thenetwork 12.

The reception unit 34 receives the video information, the environmentalinformation, and the sound information of the environmental sound of theconnection destination transmitted from the telecommunication device 11of the connection destination via the network 12 and supplies thereceived information to the environmental information analysis unit 42.

The transmission unit 33 and the reception unit 34 are configured by acommunication module or the like corresponding to wireless or wiredcommunication such as a wireless local area network (LAN) or cellularcommunication (for example, LTE-Advanced or 5G).

The output unit 35 is configured by, for example, a sound output devicesuch as a display device such as the display 22 and a sound outputdevice such as the speakers 27-1 and 27-2 in FIG. 2. The videoinformation and the sound information of the environmental sound of theconnection destination are supplied from the environmental informationanalysis unit 42 to the output unit 35. The sound information of theindication sound is supplied from the indication sound control unit 44to the output unit 35.

Based on the video information of the connection destination, the outputunit 35 causes the display 22 to display a video of the space where thetelecommunication device 11 of the connection destination is installed.Based on the sound information of the environmental sound of theconnection destination, the output unit 35 outputs the environmentalsound of the space where the telecommunication device 11 of theconnection destination is installed from the speakers 27-1 and 27-2.Further, the output unit 35 outputs the indication sound from thespeakers 27-1 and 27-2 under the control of the indication sound controlunit 44.

The configuration of the telecommunication device 11 illustrated in FIG.6 is exemplary and another configuration may be used. For example, astorage unit or the like serving as an auxiliary storage deviceconfigured by a semiconductor memory, a hard disk drive (HDD), or thelike may be provided. In the output unit 35, a display device such as aprojector may be used instead of the display 22, or a sound outputdevice such as an earphone connected in a wired or wireless manner maybe used instead of the speakers 27-1 and 27-2.

The indication sound control unit 44 may perform notification to theuser using another scheme without being limited to the indication sound.That is, apart from the indication sound (sound), a scheme of causing anilluminant such as a light emitting diode (LED) to emit light, causingthe display 22 to display a graphical user interface (GUI), orperforming wireless communication with a device carried by the user andcausing the device to vibrate can also be used. A plurality of schemesmay be used in combination. In this way, the indication sound controlunit 44 controls the output unit 35 that has various output functions,so to speak, as an output control unit such that the user is notified.

Operation of Telecommunication Device 11

Here, an operation of the telecommunication device 11 that has theforegoing configuration will be described with reference to theflowchart of FIG. 7.

Processing to be described with reference to FIG. 7 is started, forexample, when the video information and the sound information of theenvironmental sound are transmitted from the telecommunication device 11of the connection destination via the network 12.

Processing of the degree-of-interest analysis unit 41 that analyzes thedegree of interest of the user outside of the angle of camera field ofthe connection destination is appropriately performed in parallel to theprocessing of FIG. 7. The transmission unit 33 transmits the videoinformation, the environmental information, and the sound information ofthe environmental sound acquired by the input unit 31 to thetelecommunication device 11 of the connection destination via thenetwork 12.

In step S1, the reception unit 34 receives the video information, theenvironmental information, and the sound information of theenvironmental sound of the connection destination transmitted from thetelecommunication device 11 of the connection destination.

In step S2, the environmental information analysis unit 42 analyzes thesound heard from within the angle of camera field and the sound heardfrom outside of the angle of camera field, which are included in theenvironmental sound of the connection destination, based on theenvironmental information of the connection destination.

In step S3, the indication sound output method selection unit 43 selectsan output method for the indication sound in accordance with thedegree-of-interest information analyzed by the degree-of-interestanalysis unit 41.

In step S4, the indication sound control unit 44 generates the soundinformation of the indication sound by processing the sound informationof the environmental sound of the connection destination. As describedabove, the generation of the sound information of the indication soundis processing performed when there is a user who performs an action ofinterest of paying attention to the person outside of the angle ofcamera field.

In step S5, the indication sound control unit 44 causes the output unit35 to output the indication sound by the output method selected by theindication sound output method selection unit 43 along with theenvironmental sound and the video of the connection destination.

Through the foregoing processing, the telecommunication device 11 causesonly the user paying attention to the person outside of the angle ofcamera field of the connection destination to be aware of presence of aperson outside of the angle of camera field of the connectiondestination.

The user of the telecommunication device 11 can receive a notificationof information indicating presence of a person outside of the angle ofcamera field of the connection destination when the user performs anaction of being interested in the person outside of the angle of camerafield of the connection destination.

3. Second Output Control

Overview of Output Control

FIGS. 8 and 9 are diagrams illustrating an overview of second outputcontrol of the telecommunication device 11.

In this example, output of the indication sound is controlled inaccordance with the degree of interest of the user outside of the angleof camera field of the connection destination rather than the degree ofinterest of the user viewing the video of the connection destination.

In the example of FIG. 8, the user A2 is in front of thetelecommunication device 11 a of the site A as in the case describedwith reference to FIG. 4. On the other hand, it is assumed that the userB2 is within range #3 of the site B. As indicated by a balloon of FIG.8, the user B2 is not interested in the video of the site A.

In this way, when the user B2 who is a user outside of the angle ofcamera field is not interested in the video of the site A, thetelecommunication device 11 a performs an output in a normal videocommunication system.

As indicated by the balloon of FIG. 9, when the user B2 is interested inthe video of the site A, the telecommunication device 11 a outputs theemphasized indication sound toward the user A2 interesting the user B2.

FIG. 10 is a diagram illustrating examples of combinations of situationsof a B site and output method on the site A side.

In the example of FIG. 10, in a descending order of the degree ofinterest of the user B2, a situation in which the user B2 is outside ofthe angle of camera field and the degree of interest of the user B2 ishigh, a situation in which the user B2 is outside of the angle of camerafield and the degree of interest of the user B2 is less than apredetermined degree of interest despite being an interest, a situationin which the user B2 is outside of the angle of camera field but thereis no interest, and a situation in which the user B2 is not outside ofthe angle of camera field are shown as situations of the site B.

As shown in the second row of the lower column of the item in the firstrow of the table of FIG. 10, when the situation of the site B is asituation in which the user B2 is outside of the angle of camera fieldand the degree of interest of the user B2 is high, the telecommunicationdevice 11 a adjusts directivity of the speakers 27-1 and 27-2 andoutputs an emphasized indication sound toward the user A2.

When the user B2 gazes at the user A2 near the telecommunication device11 b, the visual line is oriented to the user A2 for a predeterminedtime or more, the visual line is oriented a predetermined number oftimes or more, or the like, the telecommunication device 11 a determinesthat the degree of interest of the user B2 is high.

As illustrated in the third row of the table of FIG. 10, when thesituation of the site B is a situation in which the user B2 is outsideof the angle of camera field and pays attention to the user A2, but thedegree of interest is less than a threshold, the telecommunicationdevice 11 a adjusts the directivity of the speakers 27-1 and 27-2 andoutputs the indication sound toward the user A2.

As illustrated in the fourth row of the table of FIG. 10, when thesituation of the site B is a situation in which the user B2 is outsideof the angle of camera field but is not interested in the video of thesite A, the telecommunication device 11 a performs an output in a normalvideo communication system.

As illustrated in the fifth row of the table of FIG. 10, when thesituation of the site B is a situation in which the user is not outsideof the angle of camera field, the telecommunication device 11 a performsan output in the normal video communication system.

In this way, as the degree of interest of the user B2 is higher, theindication sound heard from outside of the angle of camera field isoutput toward the user A2 in a more emphasized form.

Thus, the user A2 can be aware of a person who is interested in the userA2 herself or himself outside of the angle of camera field of thetelecommunication device 11 b (the camera 24 installed in thetelecommunication device 11 b) of the connection destination by theindication sound output from the telecommunication device 11 a.

Configuration of Telecommunication Device 11

FIG. 11 is a block diagram illustrating a second exemplary configurationof the telecommunication device 11.

In FIG. 11, the same reference numerals are given to the sameconfigurations as the configurations of the telecommunication device 11of FIG. 6. The repeated description will be appropriately omitted. Thesame applies to FIG. 16 to be described below.

The configuration of the processing unit 32 illustrated in FIG. 11 isdifferent from the configuration described with reference to FIG. 6 inthat a connection destination degree-of-interest analysis unit 51 isprovided at the rear stage of the environmental information analysisunit 42.

The information indicating the analysis result by the environmentalinformation analysis unit 42, the video information, the environmentalinformation, and the sound information of the environmental sound of theconnection destination are supplied from the environmental informationanalysis unit 42 to the connection destination degree-of-interestanalysis unit 51.

The connection destination degree-of-interest analysis unit 51 suppliesthe output unit 35 with the video information and the sound informationof the environmental sound of the connection destination supplied fromthe environmental information analysis unit 42.

The connection destination degree-of-interest analysis unit 51 analyzesthe degree of interest of the user outside of the angle of camera fieldof the connection destination based on the environmental information ofthe connection destination supplied from the environmental informationanalysis unit 42.

The connection destination degree-of-interest analysis unit 51 analyzesmagnitude of the degree of interest of the user of the connectiondestination, for example, based on a position at which the user of theconnection destination is gazing, a length of a time in which a visualline is oriented to the display 22, the number of times the visual lineis oriented to the display 22, and an orientation of the head of a userof the connection destination.

For example, the degree of interest of the user outside of the angle ofcamera field of the connection destination is analyzed by eye-trackingbased on the information acquired by an infrared sensor of theconnection destination. Based on a video in which the aspect outside ofthe angle of camera field of the connection destination is shown, thedegree of interest of the user outside of the angle of camera field ofthe connection destination may be analyzed by a method of detecting theorientation of the face of the user of the connection destination.

The information acquired by the infrared sensor and the video in whichthe aspect outside of the angle of camera field of the connectiondestination is shown, which are used to analyze the degree of interestof the user of the connection destination, are information included inthe environmental information of the connection destination.

The degree-of-interest information of the user of the connectiondestination which is information indicating the degree of interest ofthe user of the connection destination analyzed by the connectiondestination degree-of-interest analysis unit 51 is supplied to theindication sound output method selection unit 43 along with the soundinformation of the environmental sound of the connection destination andthe information indicating the result of the analysis by theenvironmental information analysis unit 42.

The indication sound output method selection unit 43 selects a method ofoutputting the indication sound based on the degree-of-interestinformation of the user of the connection destination supplied from theconnection destination degree-of-interest analysis unit 51. For example,the degree of interest of the user of the connection destination isdetermined based on a threshold and the output method for the indicationsound is selected based on a determination result.

Operation of Telecommunication Device 11

Here, an operation of the telecommunication device 11 that has theconfiguration of FIG. 11 will be described with reference to theflowchart of FIG. 12.

Processing to be described with reference to FIG. 12 is started, forexample, after the video information and the sound information of theenvironmental sound are transmitted from the telecommunication device 11of the connection destination via the network 12.

Processing of the connection destination degree-of-interest analysisunit 51 that analyzes the degree of interest of the user outside of theangle of camera field of the connection destination is appropriatelyperformed in parallel to the processing of FIG. 12. The transmissionunit 33 transmits the video information, the environmental information,and the sound information of the environmental sound acquired by theinput unit 31 to the telecommunication device 11 of the connectiondestination via the network 12.

The processing of steps S51 and S52 is the same as the processing ofsteps S1 and S2 of FIG. 7. That is, the sound heard from within theangle of camera field and the sound heard from outside of the angle ofcamera field, which are included in the environmental sound of theconnection destination, are analyzed.

In step S53, the indication sound output method selection unit 43selects an output method for the indication sound in accordance with thedegree-of-interest information of the user of the connection destinationanalyzed by the connection destination degree-of-interest analysis unit51.

The processing of steps S54 and S55 of FIG. 7 is the same as theprocessing of steps S4 and S5. That is, the sound information of theindication sound is generated, and the indication sound is output alongwith the environmental sound and the video of the connectiondestination.

Through the foregoing processing, the telecommunication device 11 causesonly the user interesting the person outside of the angle of camerafield of the connection destination to be aware of presence of a personoutside of the angle of camera field.

4. Third Output Control

FIGS. 13 and 14 are diagrams illustrating an overview of third outputcontrol of the telecommunication device 11.

In this example, output of the indication sound which is a surroundingsound of the user outside of the angle of camera field is controlled inaccordance with the degree of interest of the user outside of the angleof camera field of the connection destination.

In the example of FIG. 13, the user A2 is in front of thetelecommunication device 11 a of the site A as in the case describedwith reference to FIG. 9. On the other hand, it is assumed that the userB2 is within range #3 of the site B. As indicated by a balloon of FIG.13, the user B2 is interested in the video of the site A.

In this way, when the user B2 who is a user outside of the angle ofcamera field of the connection destination is interested in the video ofthe connection destination, the telecommunication device 11 a outputsthe emphasized indication sound toward the user A2 in whom the user B2is interested. In this example, the sound heard from range #11 which isa surrounding range of the user B2 is output as an indication sound.

As illustrated in FIG. 14, when the user B2 who is interested in user A2is within range #1, the telecommunication device 11 a outputs the soundheard from range #11 as an indication sound.

Since a flow of the processing and the configuration of thetelecommunication device 11 in the third output control are basicallysimilar to those of the second output control, that is, theconfiguration illustrated in FIG. 11 and the flow of the processingillustrated in FIG. 12, description thereof will be omitted.

As described above, the user A2 can be aware of a person who isinterested in the user A2 herself or himself outside of the angle ofcamera field or within the angle of camera field by the indication soundoutput from the telecommunication device 11 a.

5. Fourth Output Control

Overview of Output Control

FIG. 15 is a diagram illustrating an overview of fourth output controlof the telecommunication device 11.

In this example, output of the sound in the telecommunication device 11b is controlled in accordance with the secrecy of an operation of a userwho is in a space where the telecommunication device 11 a is installed.

In the example of FIG. 15, it is assumed that users A11 and A12 who aretwo users are in front of the telecommunication device 11 a of the siteA. On the other hand, in the site B, it is assumed that only the user B2is within range #3. As indicated by the balloon of FIG. 15, the usersA11 and A12 are talking about content with the high secrecy such asprivate content without being aware of presence of the user B2 outsideof the angle of camera field of the site B.

In such a situation, when a talk with high secrecy between the users A11and A12 is detected in the site A, the telecommunication device 11 bprocesses a secret sound collected in the site A based on the secrecy ofthe talk between the users A11 and A12 to degrade the quality of thesound and outputs the degraded sound. The secret sound is a sound suchas a secret talk in the sound included in the environmental sound.

The user B2 hears the sound of talk content which is hard to hearbecause of degradation of the sound quality.

Thus, the telecommunication device 11 b can output the sound of the talkhaving high secrecy between the users near the telecommunication device11 a of the connection destination with quality of the sound which ishard to hear to the user in the site B.

In the example of FIG. 15, the case in which the telecommunicationdevice 11 b detects the talk having high secrecy between the users A11and A12 in the site A has been described. However, when a talk havinghigh secrecy among a plurality of users in the site B is detected, thetelecommunication device 11 a can also perform similar processing.

Configuration of Telecommunication Device 11

FIG. 16 is a block diagram illustrating a third exemplary configurationof the telecommunication device 11.

The configuration of the processing unit 32 illustrated in FIG. 16 isdifferent from the configuration described with reference to FIG. 6 inthat a secrecy analysis unit 71, a connection destination secrecyacquisition unit 72, a secret sound output method selection unit 73, anda secret sound control unit 74 are provided.

The video information, the environmental information, and the soundinformation of the environmental sound are supplied from the input unit31 to the secrecy analysis unit 71.

The secrecy analysis unit 71 analyzes the secrecy of a sound of a userin the space where the telecommunication device 11 is installed based onthe environmental information and the sound information of theenvironmental sound supplied from the input unit 31.

The secrecy analysis unit 71 determines that, for example, a sound of atalk having content unknown to the world or a sound of a talk havingcontent which can hurt other people as a sound having high secrecy.

In the analysis of the level of the secrecy, a learning model generatedthrough machine learning in accordance with a scheme such as a neuralnetwork or deep learning may be used.

Secrecy information which is information indicating secrecy of the soundof the user analyzed by the secrecy analysis unit 71 is supplied to thetransmission unit 33 along with the video information, the environmentalinformation, and the sound information of the environmental soundsupplied from the input unit 31.

The transmission unit 33 transmits the video information, theenvironmental information, the sound information of the environmentalsound, and the secrecy information supplied from the secrecy analysisunit 71 to the telecommunication device 11 of the connection destinationvia the network 12.

The reception unit 34 receives the video information, the environmentalinformation, the sound information of the environmental sound, and thesecrecy information of the connection destination transmitted from thetelecommunication device 11 of the connection destination via thenetwork 12 and supplies them to the environmental information analysisunit 42 of the processing unit 32.

The environmental information analysis unit 42 supplies informationindicating a result of the analysis by the environmental informationanalysis unit 42 to the connection destination secrecy acquisition unit72 along with the video information of the connection destination, thesound information of the connection destination, and the secrecyinformation of the connection destination.

The connection destination secrecy acquisition unit 72 supplies theoutput unit 35 with the video information and the sound information ofthe environmental sound of the connection destination supplied from theenvironmental information analysis unit 42.

The connection destination secrecy acquisition unit 72 supplies thesecret sound output method selection unit 73 with the secrecyinformation and the sound information of the environmental sound of theconnection destination supplied from the environmental informationanalysis unit 42 and the information indicating the result of theanalysis by the environmental information analysis unit 42.

The secret sound output method selection unit 73 selects a method ofoutputting the secret sound based on the secrecy information suppliedfrom the connection destination secrecy acquisition unit 72 and theinformation indicating the result of the analysis by the environmentalinformation analysis unit 42. For example, the secret sound outputmethod selection unit 73 selects a method of distorting the secret soundor a method of raising a level of only a specific frequency.

Information indicating the method of outputting the secret sound issupplied to the secret sound control unit 74 along with the secrecyinformation and the sound information of the environmental sound of theconnection destination.

The secret sound control unit 74 generates the sound information of thesecret sound by processing the sound information of the environmentalsound of the connection destination based on the secrecy information ofthe connection destination supplied from the secret sound output methodselection unit 73. For example, the secret sound control unit 74generates the sound information of the secret sound by extracting soundinformation of a sound of a talk having high secrecy from the soundinformation of the environmental sound of the connection destination.

The secret sound control unit 74 processes the secret sound based oninformation indicating the method of outputting the secret soundsupplied from the secret sound output method selection unit 73. Thesecret sound control unit 74 supplies the sound information of theprocessed secret sound to the output unit 35 to output the processedsecret sound.

Operation of Telecommunication Device 11

Here, an operation of the telecommunication device 11 that has theconfiguration of FIG. 16 will be described with reference to theflowchart of FIG. 17.

Processing to be described with reference to FIG. 17 is started, forexample, when the video information, the sound information of theenvironmental sound, and the secrecy information are transmitted fromthe telecommunication device 11 of the connection destination via thenetwork 12.

The transmission unit 33 transmits the video, the sound information ofthe environmental sound, and the secrecy information acquired by theinput unit 31 to the telecommunication device 11 of the connectiondestination via the network 12.

In step S101, the reception unit 34 receives the video information, thesound information of the environmental sound, and the secrecyinformation of the connection destination transmitted from thetelecommunication device 11 of the connection destination.

In step S102, the environmental information analysis unit 42 and theconnection destination secrecy acquisition unit 72 supplies the soundinformation of the environmental sound of the connection destination tothe secret sound output method selection unit 73 along with the secrecyinformation and information indicating a result of the analysis by theenvironmental information analysis unit 42.

In step S103, the secret sound output method selection unit 73 selectsthe method of outputting the secret sound based on the secrecyinformation of the connection destination.

In step S104, the secret sound control unit 74 generates the secretsound by processing the sound information of the environmental sound ofthe connection destination based on the secrecy information of theconnection destination. The secret sound control unit 74 processes thesecret sound based on information indicating the method of outputtingthe secret sound. As described above, the generation of the secret soundand the processing of the secret sound are processing performed whenthere are users performing a talk having high secrecy.

In step S105, the secret sound control unit 74 causes the output unit 35to output the processed secret sound along with the environmental soundand the video of the connection destination.

Through the foregoing processing, the sound of the talk having the highsecrecy can be prevented from being presented to a user outside of theangle of camera field of the telecommunication device 11 of theconnection destination.

A surrounding indication sound of the user outside of the angle ofcamera field of the connection destination may be output toward userswho are performing an action having high secrecy so that the users canbe aware of presence of the user outside of the angle of camera field ofthe connection destination.

FIG. 18 is a diagram illustrating an overview of another output controlperformed by the telecommunication device 11.

In the example of FIG. 18, it is assumed that the user A11 is in frontof the telecommunication device 11 a of the site A. On the other hand,it is assumed that only the user B2 is within range #3 of the site B.The user A11 is a user who is performing an action having high secrecy,such as a private action, without being aware of the user B2 outside ofthe angle of camera field.

When it is detected that the user A11 is performing the action havinghigh secrecy, the telecommunication device 11 a outputs an emphasizedindication sound toward the user A11. The output of the indication soundoutside of the angle of camera field of the connection destination iscontrolled in accordance with a level of the secrecy of the actionperformed by the user A11 who is in the site A. The user A11 may benotified in accordance with another scheme without being limited to theindication sound.

Since the indication sound is output toward the user A11 performing theaction having the high secrecy, the telecommunication device 11 cancause only the user A11 to be aware of presence of the person outside ofthe angle of camera field.

For example, the user A11 who is aware of presence of the person outsideof the angle of camera field of the connection destination can stop theaction having the high secrecy.

In this way, by presenting the indication sound to only the user who isperforming the action having the high secrecy, the user can be aware ofthe presence of the person outside of the angle of camera field in amore appropriate form.

6. Modification Examples

Illuminant

Presence of a person outside of an angle of camera field of a connectiondestination may be presented using luminescence of an illuminantprovided in the telecommunication device 11.

FIG. 19 is a front view illustrating an example of the telecommunicationdevice 11 a in which an illuminant is provided.

For example, an illuminant is provided on an upper, lower, left, or leftedge of the display 22 of the telecommunication device 11 a.

When the user B2 is outside of the angle of camera field of thetelecommunication device 11 b, as illustrated in FIG. 19, a video of theuser B2 is not shown on the display 22 of the telecommunication device11 a. In FIG. 19, the dotted line of the user B2 to the right of thetelecommunication device 11 a indicates that the user B2 is outside ofthe angle of camera field in the site B.

In this case, the telecommunication device 11 a causes an illuminantprovided on a right edge portion, which is an illuminant for an edge ina direction in which there is the user B2, to emit light. In FIG. 19, aportion in which the right end of the display 22 is colored indicatesthat the illuminant emits light.

When there is only one user around the telecommunication device 11 a orthe presence of a person outside of the angle of camera field issimultaneously delivered to a plurality of users around thetelecommunication device 11 a, light is emitted to suggest the presenceof a person outside of the angle of camera field. The intensity of lightof the illuminant may indicate the degree of interest of the user B2 ina video of the site A.

FIG. 20 is a diagram illustrating an example of relation betweenintensity of light and the degree of interest.

In FIG. 20, the horizontal axis represents the degree of interest of theuser B2, and the vertical axis represents intensity of light.

As illustrated in FIG. 20, the telecommunication device 11 a causesstrong light to be emitted, for example, as the degree of interest ofthe user B2 outside of an angle of camera field of a connectiondestination is higher.

A user of the telecommunication device 11 a can know the degree ofinterest of a user outside of the angle of camera field of theconnection destination in accordance with the intensity of light fromthe illuminant.

The degree of interest of the user outside of the angle of camera fieldof the connection destination is notified of by causing a portableterminal such as a smartphone carried by the user of thetelecommunication device 11 a to vibrate. In this case, the indicationsound control unit 44 of the telecommunication device 11 a controls theportable terminal carried by the user of the telecommunication device 11a such that the portable terminal vibrates in accordance with the degreeof interest of the user B2 outside of the angle of camera field of theconnection destination.

Camera 24 and Sensor 25

The camera 24 and the sensor 25 provided in the sensor unit 23 have beendescribed above. However, the camera 24 and the sensor 25 may beprovided in another form.

FIG. 21 is a diagram schematically illustrating examples of forms of thecamera 24 and the sensor 25.

As illustrated in A of FIG. 21, the camera 24 and the sensor 25 can beseparately provided. In the example of A of FIG. 21 the sensors 25-1 and25-2 configured as the cameras are provided at left and right ends of aframe 21.

A video with range #51 which is a range of an angle of field of thecamera 24 is displayed on the display 22 of the telecommunication device11 of a connection destination. Videos with ranges #52-1 and #52-2 areused to detect an aspect outside of an angle of display field of thedisplay (outside of an angle of field of the camera 24).

Ranges #52-1 and #52-2 are a range of the angles of field of the sensors25-1 and 23-2, excluding a range overlapping range #51.

As illustrated in B of FIG. 21, the camera 24 can also be used as thesensor 25. For example, the camera 24 is configured by a wide-anglecamera and is provided in the middle of the telecommunication device 11.

A video with range #61 indicating a range of a part of a range of theangle of field of the camera 24 is displayed on the display 22 of thetelecommunication device 11 of the connection destination. A video withrange #62 is used to detect an aspect outside of the angle of displayfield of the display.

Range #62 is a range of an angle of field of the camera 24 excludingrange #61.

As illustrated in C of FIG. 21, the sensor 25 may be provided outside ofthe telecommunication device 11. For example, a camera configured as thesensor 25 is provided on a wall different from a wall on which thetelecommunication device 11 is installed.

As in A of FIG. 21, a video with range #51 which is a range of the angleof field of the camera 24 is displayed on the display 22 of thetelecommunication device 11 of the connection destination. A video withrange #71 is used to detect an aspect outside of the angle of displayfield of the display.

Range #71 is a range of the angle of field of the sensor 25 excluding aregion overlapping range #51.

System Configuration

In FIG. 6 and the like, the processing unit 32 is provided in thetelecommunication device 11. Some or all of the functions of theprocessing unit 32 may be implemented by a server connected to thenetwork 12.

FIG. 22 is a diagram illustrating another exemplary configuration of thevideo communication system.

The video communication system illustrated in FIG. 22 is configured suchthat the telecommunication device 11 a, the telecommunication device 11b, and a server 101 are connected to each other via the network 12 suchas the Internet. In the server 101, a configuration of a part or all ofthe processing unit 32 described with reference to FIG. 6, 11, or 16 isprovided.

In this way, the processing unit 32 may be provided outside of thetelecommunication device 11.

For example, various kinds of information such as video information,environmental information, and sound information such as anenvironmental sound are transmitted from the telecommunication device 11to the server 101.

For example, various kinds of information such as video information,environmental information, and sound information such as anenvironmental sound of a connection destination are transmitted from theserver 101 to the telecommunication device 11.

A home server controlling the telecommunication device 11 may beprovided for telecommunication device 11.

FIG. 23 is a diagram illustrating still another exemplary configurationof the video communication system.

A video communication system illustrated in FIG. 23 is configured suchthat the telecommunication device 11 a and the telecommunication device11 b are connected via the network 12.

The telecommunication device 11 a includes an input/output unit 111 aand a home server 112 a. In the input/output unit 111 a, theconfigurations of the input unit 31 and the output unit 35 are provided.In the home server 112 a, the configurations of the processing unit 32,the transmission unit 33, and the reception unit 34 are provided. Thehome server 112 a is connected to the home server 112 b of thetelecommunication device 11 b via the network 12.

Similarly, the telecommunication device 11 b includes an input/outputunit 111 b and a home server 112 b. In the input/output unit 111 b, theconfigurations of the input unit 31 and the output unit 35 are provided.In the home server 112 b, the configurations of the processing unit 32,the transmission unit 33, and the reception unit 34 are provided. Thehome server 112 b is connected to the home server 112 a via the network12.

In this way, the configuration of the processing unit 32 or the like maybe provided outside of the input/output unit 111 including the display22 or the like.

A partial configuration of the processing unit 32 may be provided in thehome servers 112 a and 112 b and the remaining configuration of theprocessing unit 32 may be provided in the input/output unit 111 a and111 b.

As described above, in the present technology, when, between a space ofthe site A at which the telecommunication device 11 a is installed and aspace of the site B at which the telecommunication device 11 b isinstalled, a video of captured images captured by the camera 24 in onespace is displayed in real time by the display 22 in the other space,processing is performed to notify a user at the site A that there is theuser when the user at a position outside of an angle of field of thecamera 24 installed at the site B is detected.

Thus, the user viewing the video of the site B at the site A can beallowed to be aware of presence of the person outside of the angle offield of the video of the site B in a more appropriate form.

In particular, in the present technology, processing is performed tonotify a user who is a specific target among users at the site A of anindication sound, which is included in sounds collected by themicrophone 26 installed at the site B outside of the angle of field ofthe camera 24 installed at the site B.

Thus, when there are a plurality of users viewing the aspect of the siteB at the site A, only some users who pay attention to the person outsideof the angle of camera field can be aware of presence of a personoutside of the angle of camera field.

Example of Computer

Each processing described with reference to the above-describedflowchart may not necessarily be performed chronologically in the orderdescribed in the flowchart and includes processing performed in parallelor individually (for example, parallel processing or processing inaccordance with an object). A program may be processed by a single CPUor may be distributed and processed by a plurality of CPUs.

The series of steps of processing by the above-describedtelecommunication device 11 and the like can be performed by hardware orsoftware. When the series of steps of processing are performed bysoftware, a program of the software is installed from a programrecording medium that records the program to a computer embedded indedicated hardware or, for example, a general-purpose personal computercapable of performing various functions by installing the variousprograms.

FIG. 24 is a block diagram illustrating an exemplary hardwareconfiguration of a computer that performs the above-described series ofsteps of processing in accordance with a program.

In the computer, a CPU 201, a ROM 202, and a RAM 203 are connected toeach other via a bus 204.

An input/output interface 205 is further connected to the bus 204. Aninput unit 206 such as a keyboard, a mouse, or a microphone, an outputunit 207 such as a display or a speaker, a storage unit 208 such as ahard disk or a nonvolatile memory, a communication unit 209 such as anetwork interface, and a drive 210 driving a removable medium 211 suchas a magnetic disk, an optical disc, a magneto-optical disk, or asemiconductor memory are connected to the input/output interface 205.

In the computer that has the foregoing configuration, the CPU 201performs the foregoing series of steps of processing, for example, byloading the program stored in the storage unit 208 to the RAM 203 viathe input/output interface 205 and the bus 204 and executing theprogram.

For example, the program executed by the computer (the CPU 201) isrecorded on the removable medium 211 which is a package medium such as amagnetic disk (including a flexible disk), an optical disc (a compactdisc-read only memory (CD-ROM), a digital versatile disc (DVD), or thelike), a magneto-optical disk, or a semiconductor memory, or is providedvia a wired or wireless transmission medium such as a local areanetwork, the Interest, or digital satellite broadcasting.

The program can be installed to the storage unit 208 via theinput/output interface 205 by mounting the removable medium 211 on thedrive 210. The program can be received by the communication unit 209 viaa wired or wireless transmission medium to be installed in the storageunit 208. Further, the program can be installed in advance in the ROM202 or the storage unit 208.

Others

In the present specification, the system means a set of a plurality ofconstituent elements (devices, modules (components), or the like) andall the constituent elements may be included or not included in the samecasing. Accordingly, a plurality of devices accommodated in separatecasings and connected via a network and one device in which a pluralityof modules are accommodated in one casing may all be a system.

The advantageous effects described in the present specification aremerely exemplary and are not restrictive and other advantageous effectsmay be realized.

Embodiments of the present technology are not limited to theabove-described embodiments and can be modified in various forms withinthe scope of the present technology departing from the gist of thepresent technology.

For example, the present technology may have a configuration of cloudingcomputing in which a plurality of devices share and process one functiontogether via a network.

<Combination Examples of Configurations>

The present technology can be configured as follows.

(1)

An information processing device including a processing unit,

wherein, between a first space where a first imaging device and a firstdisplay device are installed and a second space where a second imagingdevice and a second display device are installed, when a captured imagecaptured by the imaging device in one of the spaces is displayed by thedisplay device in the other space in real time,the processing unit performs processing to notify first users in thefirst space that second users are outside of an angle of field of thesecond imaging device when the second users who are at positions outsideof the angle of field of the second imaging device are detected.(2)

The information processing device according to (1), wherein theprocessing unit performs processing to notify a first specific user whois a specific target among the first users in the first space of anindication sound outside of the angle of field of the second imagingdevice included in sounds collected in the second space.

(3)

The information processing device according to (2),

wherein the processing unit includesan output control unit that outputs the sounds collected in the secondspace to a first sound output device installed in the first space inreal time, andan analysis unit that analyzes the indication sound outside of the angleof field of the second imaging device among the sounds collected in thesecond space, and wherein the output control unit outputs the indicationsound to the first sound output device in real time based on an analysisresult of the indication sound.(4)

The information processing device according to (3),

wherein the processing unit further includes an output method selectionunit that selects an output method for the indication sound based on theanalysis result of the indication sound, andwherein the output control unit causes the first sound output device tooutput the indication sound using an output method for a selectiontarget.(5)

The information processing device according to (4),

wherein the processing unit further includes a first degree-of-interestanalysis unit that analyzes the degree of interest of a first interesteduser who is present at the first space and interested in an outside ofthe angle of field of the second imaging device with respect to theoutside of the angle of field of the second imaging device, andwherein the output method selection unit selects an output method forthe indication sound in accordance with the degree of interest of thefirst interested user.(6)

The information processing device according to (5), wherein the outputcontrol unit causes the indication sound to be output to the firstinterested user among the first users in the first space.

(7)

The information processing device according to (5) or (6), wherein thefirst degree-of-interest analysis unit analyzes the degree of interestof the first interested user based on a position which is outside of theangle of field of the second imaging device and to which a visual lineof the first interested user is oriented, a length of a time in whichthe visual line of the first interested user is oriented outside of theangle of field of the second imaging device, an orientation of the headof the first interested user, and a sound spoken by the first interesteduser.

(8)

The information processing device according to (6),

wherein the output method selection unit selects a method of adjustingand outputting directivity of the indication sound as the output methodfor the indication sound, andwherein the output control unit causes the indication sound of which thedirectivity is adjusted to be output to the first interested user.(9)

The information processing device according to (4),

wherein the processing unit further includes a second degree-of-interestanalysis unit that analyzes the degree of interest of the second usersat a position outside of the angle of field of the second imaging devicein the second space with respect to the first space, andwherein the output control unit selects an output method for theindication sound in accordance with the degree of interest of a secondinterested user interested in the first space.(10)

The information processing device according to (9), wherein the outputcontrol unit causes the indication sound to be output to a first user ofinterest in whom the second interested user is interested among thefirst users in the first space.

(11)

The information processing device according to (10), wherein the seconddegree-of-interest analysis unit analyzes the degree of interest of thesecond interested user based on a position of the first space to which avisual line of the second interested user is oriented, a length of atime in which the visual line of the second interested user is orientedto the first user of interest, the number of times the visual line ofthe second interested user is oriented to the first user of interest,and an orientation of the head of the second interested user.

(12)

The information processing device according to (10),

wherein the output method selection unit selects a method of adjustingand outputting directivity of the indication sound as the output methodfor the indication sound, andwherein the output control unit causes the indication sound of which thedirectivity is adjusted to be output to the first user of interest.(13)

The information processing device according to any one of (9) to (12),wherein the output control unit causes the first sound output device tooutput a surrounding sound of the second interested user as anindication sound.

(14)

The information processing device according to (9), wherein the outputcontrol unit causes an illuminant installed in the first space to emitlight in accordance with the degree of interest of the second interesteduser.

(15)

The information processing device according to (9), wherein the outputcontrol unit causes a device carried by the first user of interest inwhom the second interested user is interested to vibrate among the firstusers in the first space in accordance with the degree of interest ofthe second interested user.

(16)

The information processing device according to (1),

wherein the processing unit includesan output control unit that causes the first sound output deviceinstalled in the first space to output sounds collected in the secondspace in real time, and a secrecy acquisition unit that acquires secrecyof actions performed by the second users in the second space, andwherein the output control unit causes a secret sound to be output tothe first users in the first space in accordance with the secrecy.(17)

The information processing device according to (1), wherein theprocessing unit includes

an output control unit that causes the first sound output deviceinstalled in the first space to output sounds collected in the secondspace in real time, and a secrecy analysis unit that analyzes secrecy ofactions performed by the first users in the first space, andwherein the output control unit causes an indication sound to be outputto the first users in the first space in accordance with the secrecywhen the second users are outside of the angle of field of the secondimaging device in the second space.(18)

The information processing device according to any one of (1) to (17),wherein the first imaging device and the first display device installedin the first space and the second imaging device and the second displaydevice installed in the second space are integrated and are connected toan information processing device installed in the other space via anetwork.

(19)

An information processing method,

wherein, between a first space where a first imaging device and a firstdisplay device are installed and a second space where a second imagingdevice and a second display device are installed, when a captured imagecaptured by the imaging device in one of the spaces is displayed by thedisplay device in the other space in real time,an information processing device performs processing to notify firstusers in the first space of a sound heard from outside of the angle offield of the second imaging device when the sound heard from outside ofthe angle of field of the second imaging device is detected among soundscollected in the second space.(20)

A program causing a computer to perform,

between a first space where a first imaging device and a first displaydevice are installed and a second space where a second imaging deviceand a second display device are installed, when a captured imagecaptured by the imaging device in one of the spaces is displayed by thedisplay device in the other space in real time,processing to notify first users in the first space of a sound heardfrom outside of the angle of field of the second imaging device when thesound heard from outside of the angle of field of the second imagingdevice is detected among sounds collected in the second space.

REFERENCE SIGNS LIST

-   1 Video communication system-   11 Telecommunication device-   21 Frame-   22 Display-   23 Sensor unit-   24 Camera-   25-1, 25-2 Sensor-   26 Microphone-   27-1, 27-2 Speaker-   31 Input unit-   32 Processing unit-   33 Transmission unit-   34 Reception unit-   35 Output unit-   41 Degree-of-interest analysis unit-   42 Environmental information analysis unit-   43 Indication sound output method selection unit-   44 Indication sound control unit-   51 Connection destination degree-of-interest analysis unit-   71 Secrecy analysis unit-   72 Connection destination secrecy acquisition unit-   73 Secret sound output method selection unit-   74 Secret sound control unit-   101 Server-   111 a, 111 b Home server-   112 a, 112 b Input/output unit

1. An information processing device comprising a processing unit,wherein, between a first space where a first imaging device and a firstdisplay device are installed and a second space where a second imagingdevice and a second display device are installed, when a captured imagecaptured by the imaging device in one of the spaces is displayed by thedisplay device in the other space in real time, the processing unitperforms processing to notify first users in the first space that secondusers are outside of an angle of field of the second imaging device whenthe second users who are at positions outside of the angle of field ofthe second imaging device are detected.
 2. The information processingdevice according to claim 1, wherein the processing unit performsprocessing to notify a first specific user who is a specific targetamong the first users in the first space of an indication sound outsideof the angle of field of the second imaging device included in soundscollected in the second space.
 3. The information processing deviceaccording to claim 2, wherein the processing unit includes an outputcontrol unit that outputs the sounds collected in the second space to afirst sound output device installed in the first space in real time, andan analysis unit that analyzes the indication sound outside of the angleof field of the second imaging device among the sounds collected in thesecond space, and wherein the output control unit outputs the indicationsound to the first sound output device in real time based on an analysisresult of the indication sound.
 4. The information processing deviceaccording to claim 3, wherein the processing unit further includes anoutput method selection unit that selects an output method for theindication sound based on the analysis result of the indication sound,and wherein the output control unit causes the first sound output deviceto output the indication sound using an output method for a selectiontarget.
 5. The information processing device according to claim 4,wherein the processing unit further includes a first degree-of-interestanalysis unit that analyzes the degree of interest of a first interesteduser who is present at the first space and interested in an outside ofthe angle of field of the second imaging device with respect to theoutside of the angle of field of the second imaging device, and whereinthe output method selection unit selects an output method for theindication sound in accordance with the degree of interest of the firstinterested user.
 6. The information processing device according to claim5, wherein the output control unit causes the indication sound to beoutput to the first interested user among the first users in the firstspace.
 7. The information processing device according to claim 5,wherein the first degree-of-interest analysis unit analyzes the degreeof interest of the first interested user based on a position which isoutside of the angle of field of the second imaging device and to whicha visual line of the first interested user is oriented, a length of atime in which the visual line of the first interested user is orientedoutside of the angle of field of the second imaging device, anorientation of the head of the first interested user, and a sound spokenby the first interested user.
 8. The information processing deviceaccording to claim 6, wherein the output method selection unit selects amethod of adjusting and outputting directivity of the indication soundas the output method for the indication sound, and wherein the outputcontrol unit causes the indication sound of which the directivity isadjusted to be output to the first interested user.
 9. The informationprocessing device according to claim 4, wherein the processing unitfurther includes a second degree-of-interest analysis unit that analyzesthe degree of interest of the second users at a position outside of theangle of field of the second imaging device in the second space withrespect to the first space, and wherein the output control unit selectsan output method for the indication sound in accordance with the degreeof interest of a second interested user interested in the first space.10. The information processing device according to claim 9, wherein theoutput control unit causes the indication sound to be output to a firstuser of interest in whom the second interested user is interested amongthe first users in the first space.
 11. The information processingdevice according to claim 10, wherein the second degree-of-interestanalysis unit analyzes the degree of interest of the second interesteduser based on a position of the first space to which a visual line ofthe second interested user is oriented, a length of a time in which thevisual line of the second interested user is oriented to the first userof interest, the number of times the visual line of the secondinterested user is oriented to the first user of interest, and anorientation of the head of the second interested user.
 12. Theinformation processing device according to claim 10, wherein the outputmethod selection unit selects a method of adjusting and outputtingdirectivity of the indication sound as the output method for theindication sound, and wherein the output control unit causes theindication sound of which the directivity is adjusted to be output tothe first user of interest.
 13. The information processing deviceaccording to claim 9, wherein the output control unit causes the firstsound output device to output a surrounding sound of the secondinterested user as an indication sound.
 14. The information processingdevice according to claim 9, wherein the output control unit causes anilluminant installed in the first space to emit light in accordance withthe degree of interest of the second interested user.
 15. Theinformation processing device according to claim 9, wherein the outputcontrol unit causes a device carried by the first user of interest inwhom the second interested user is interested to vibrate among the firstusers in the first space in accordance with the degree of interest ofthe second interested user.
 16. The information processing deviceaccording to claim 1, wherein the processing unit includes an outputcontrol unit that causes the first sound output device installed in thefirst space to output sounds collected in the second space in real time,and a secrecy acquisition unit that acquires secrecy of actionsperformed by the second users in the second space, and wherein theoutput control unit causes a secret sound to be output to the firstusers in the first space in accordance with the secrecy.
 17. Theinformation processing device according to claim 1, wherein theprocessing unit includes an output control unit that causes the firstsound output device installed in the first space to output soundscollected in the second space in real time, and a secrecy analysis unitthat analyzes secrecy of actions performed by the first users in thefirst space, and wherein the output control unit causes an indicationsound to be output to the first users in the first space in accordancewith the secrecy when the second users are outside of the angle of fieldof the second imaging device in the second space.
 18. The informationprocessing device according to claim 1, wherein the first imaging deviceand the first display device installed in the first space and the secondimaging device and the second display device installed in the secondspace are integrated and are connected to an information processingdevice installed in the other space via a network.
 19. An informationprocessing method, wherein, between a first space where a first imagingdevice and a first display device are installed and a second space wherea second imaging device and a second display device are installed, whena captured image captured by the imaging device in one of the spaces isdisplayed by the display device in the other space in real time, aninformation processing device performs processing to notify first usersin the first space that second users are outside of an angle of field ofthe second imaging device when the second users who are at positionsoutside of the angle of field of the second imaging device are detected.20. A program causing a computer to perform, when, between a first spacewhere a first imaging device and a first display device are installedand a second space where a second imaging device and a second displaydevice are installed, a captured image captured by the imaging device inone of the spaces is displayed by the display device in the other spacein real time, processing to notify first users in the first space thatsecond users are outside of an angle of field of the second imagingdevice when the second users who are at positions outside of the angleof field of the second imaging device are detected.